Image pickup display apparatus, image pickup display method, and recording medium

ABSTRACT

An image pickup display apparatus includes: a communication section that is capable of receiving a picked-up image from an image pickup unit; a display section that displays the picked-up image received by the communication section; an operation section for controlling image pickup by the image pickup unit; a state determination section that determines a state of a case in which the communication section, the display section and the operation section are housed; and a control section that receives a result of determination performed by the state determination section, to change a user interface provided by the display section and the operation section based on whether or not the case is arranged on another object.

This application claims the benefit of Japanese Application No.2014-103544 in Japan on May 19, 2014, the contents of which areincorporated by this reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image pickup display apparatus thatis capable of displaying an image from a separated image pickup unit, animage pickup display method, and a recording medium.

2. Description of Related Art

In recent years, mobile devices with a photographing function(photographing devices) such as digital cameras have become common. Someof such photographing devices have a display section and a function fordisplaying a photographed image on the display section. In addition,some of them display a menu screen on a display section to facilitatethe operation of the photographing devices. Such a display section iscommonly provided on a rear surface of a photographing device main body,and a user is able to perform a photographing operation while checking athrough image displayed on the display section on the rear surface atthe time of photographing.

In addition, in recent years, a lens-style camera, which does not have adisplay section for displaying a picked-up image but only has functionsof photographing and recording, has sometimes been employed. Alens-style camera of this type is generally attached to a smartphone, atablet PC, or the like, and photographing is controlled by using thesmartphone, tablet PC, or the like to which the lens-style camera isattached as a control device. In addition, the control device such as asmartphone, tablet PC, or the like also functions as an image pickupdisplay apparatus that displays a picked-up image from the lens-stylecamera.

Such a lens-style camera is attachable to and detachable from asmartphone, tablet PC, or the like, and is capable of photographing alsoin a state detached from the smartphone, tablet PC, or the like. In thiscase, the control device such as the smartphone, tablet PC, or the likeis connected to the lens-style camera by wireless or wire communication,and the photographing operation of the lens-style camera is possible byusing a touch panel, etc., provided on the display section.

Japanese Patent Application Laid-Open Publication No. 2012-185692proposes a technique for connecting such a smartphone, tablet PC, or thelike to a mobile computer apparatus, etc., by wire or wirelessconnection, and allowing the smartphone, tablet PC or the like to actfor a function of a mouse which is input means for the computerapparatus.

Note that the lens-style camera is used in a state attached to the imagepickup display apparatus that displays a picked-up image from thelens-style camera (attached state) in some cases, and used in a statedetached from the image pickup display apparatus (non-attached state) inother cases, and it is considered that the usage patterns of thelens-style camera are significantly different in the attached state andin the non-attached state.

SUMMARY OF THE INVENTION

An image pickup display apparatus according to the present inventionincludes a communication section that is capable of receiving apicked-up image from an image pickup unit; a display section thatdisplays the picked-up image received by the communication section; anoperation section for controlling image pickup by the image pickup unit;a state determination section that determines a state of a case in whichthe communication section, the display section and the operation sectionare housed; and a control section that receives a result ofdetermination performed by the state determination section, to change auser interface provided by the display section and the operation sectionbased on whether or not the case is arranged on another object.

Further, an image pickup display method according to the presentinvention includes: receiving, by a communication section, a picked-upimage from an image pickup unit; displaying, by a display section, thepicked-up image received by the communication section; determining, by astate determination section, a state of a case in which an operationsection for controlling image pickup by the image pickup unit, thecommunication section, and the display section are housed; and receivinga result of determination performed by the state determination section,to change a user interface provided by the display section and theoperation section, by a control section, based on whether or not thecase is arranged on another object.

Furthermore, a non-transitory computer readable recording mediumaccording to the present invention which stores a program for causing acomputer to execute procedures of receiving, by a communication section,a picked-up image from an image pickup unit; displaying, by a displaysection, the picked-up image received by the communication section,determining a state of a case in which an operation section forcontrolling image pickup by the image pickup unit, the communicationsection, and the display section are housed; and changing a userinterface provided by the display section and the operation sectionbased on whether or not the case is arranged on another object.

The above and other objects, features and advantages of the inventionwill become more clearly understood from the following descriptionreferring to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a photographing system including animage pickup display apparatus according to a first embodiment of thepresent invention.

FIG. 2 is a flowchart for describing a photographing control.

FIGS. 3A to 3C are illustrative diagrams for describing workingaccording to the first embodiment.

FIG. 4 is a chart showing an example of a correspondence relationshipbetween a user interface and respective states of the image pickupdisplay apparatus and the image pickup unit.

FIG. 5 is a block diagram showing a second embodiment of the presentinvention.

FIG. 6 is an illustrative diagram showing an appearance of the apparatusshown in FIG. 5.

FIG. 7 is a flowchart for describing a camera control in a lens-stylecamera 51 and a control of a smartphone 61.

FIG. 8 is an illustrative diagram showing an example of a picked-upimage from a built-in camera section 63.

FIG. 9 is a flowchart showing an example of a specific working flow of aconnection determination communication performed by the lens-stylecamera 51 in step S37 in FIG. 7.

FIG. 10 is a flowchart showing a flow of connection determinationperformed by a control section 65 using information obtained by theconnection determination communication.

FIG. 11A and FIG. 11B are illustrative diagrams for describingsimilarity determination of shake patterns.

FIG. 12 is a flowchart showing a working flow of a display forseparation state in step S20 in FIG. 7.

FIG. 13A and FIG. 13B are illustrative diagrams showing a state ofphotographing.

FIG. 14 is a flowchart showing an example of control in an operationpattern 3.

FIGS. 15A to 15C are illustrative diagrams for describing an example ofdisplay in the operation pattern 3.

FIGS. 16A to 16D are illustrative diagrams for describing a rotationmode.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Hereinafter, detailed description will be made on embodiments of thepresent invention with reference to drawings.

First Embodiment

FIG. 1 is a block diagram showing a photographing system including animage pickup display apparatus according to the first embodiment of thepresent invention.

In FIG. 1, an image pickup unit 10 includes a communication section 11,and an image pickup display apparatus 20 includes a communicationsection 21. For example, the image pickup unit 10 can be configured by alens-style camera, and the image pickup display apparatus 20 can beconfigured by a smartphone or a tablet PC. The communication section 11and the communication section 21 can communicate with each other. Theimage pickup unit 10 and the image pickup display apparatus 20communicate with each other with the communication sections 11, 21, tothereby capable of transmitting and receiving various kinds ofinformation. Since information terminals such as recent smartphones andtablet PCs have also an image pickup unit, the term “informationterminals” may be used instead of “image pickup display apparatus” asdescribed above.

The image pickup unit 10 includes an image pickup device not shown. Animage pickup control section 12 is configured to receive a controlsignal from the image pickup display apparatus 20 through thecommunication section 11, and to be able to control image pickup to beperformed by the image pickup device. The communication section 11 isconfigured to be able to transmit a picked-up image obtained by theimage pickup device to the communication section 21.

The image pickup display apparatus 20 includes an image pickup devicenot shown. An image pickup control section 22 is configured to be ableto control image pickup by the image pickup device in the image pickupdisplay apparatus 20. The image pickup display apparatus 20 includes adisplay section 23. The display section 23 is controlled by the controlsection 25, to display a picked-up image, a menu, and the like.

The control section 25 is capable of controlling image pickup performedby the image pickup unit 10 through the communication section 21, 11. Inaddition, the control section 25 performs predetermined image processingon the picked-up image transferred from the image pickup unit 10 throughthe communication sections 11 and 21, and thereafter can send thepicked-up image subjected to the image processing to the display section23 to cause the display section to display the picked-up image.

The image pickup display apparatus 20 is provided with an operationsection 24. The operation section 24 can be configured by switches,buttons, and the like, not shown, for example, and outputs an operationsignal based on a user operation performed on the operation section 24to the control section 25. In addition, a touch panel, not shown,provided on a display screen of the display section 23 may be employedas the operation section 24.

The touch panel can generate an operation signal corresponding to aposition which is pointed by the user with a finger on the displayscreen of the display section 23. The operation signal is supplied tothe control section 25. Thus, when the user touches on or slides thedisplay screen, the control section 25 is capable of detecting variousoperations, for example, the position touched by the user, operation forclosing and then spreading two fingers (pinch operation), the slidingoperation, the position reached by the sliding operation, the slidingdirection, a period during which the user touches the display screen,etc., and capable of performing processing corresponding to the useroperation. For example, touch operation enables various photographingoperations in the image pickup unit 10 to be performed.

In the present embodiment, the image pickup display apparatus 20includes a state determination section 26. The state determinationsection 26 can determine the state of the image pickup display apparatus20 itself. For example, the state determination section 26 is capable ofdetermining whether the image pickup display apparatus 20 is placed on atable or the like (placed state) or held by the user with a hand (heldstate). The state determination section 26 is configured by anacceleration sensor, a gyro sensor, or the like, for example, andcapable of determining whether the image pickup display apparatus 20 isin the placed state or the held state based on the detection resultobtained by any of these sensors.

In addition, the state determination section 26 may be configured to beable to determine the state of the image pickup unit 10. For example,the state determination section 26 determines whether the image pickupunit 10 is in a state disposed on a tripod stand, etc. (hereinafter,referred to as stationary state) or in a state held by the user with thehand (hereinafter, referred to as camera-held state), based on apicked-up image from the image pickup unit 10. For example, the statedetermination section 26 is capable of determining the stationary stateand the camera-held state based on the movements of the respective imageportions in the picked-up image from the image pickup unit 10.

Note that the state determination section 26 may determine the placedstate and the held state of the image pickup display apparatus 20 basedon the picked-up image obtained with the image pickup device of theimage pickup display apparatus 20, and may determine the stationarystate and the camera-held state of the image pickup unit 10 by using thedetection signal from the acceleration sensor, the gyro sensor, or thelike provided in the image pickup unit 10.

Furthermore, the state determination section 26 is configured to be ableto determine whether the image pickup unit 10 is in a state attached tothe image pickup display apparatus 20 (attached state) or in a statedetached from the image pickup display apparatus 20 (non-attachedstate). These determination results by the state determination section26 are supplied to the control section 25.

In accordance with the determination result by the state determinationsection 26, the control section 25 changes control for the display onthe display section 23 and the operation signal corresponding to thedisplay which is transmitted from the operation section 24, that is, agraphical user interface.

Next, the working in the embodiment thus configured will be describedwith reference to FIG. 2 and FIGS. 3A to 3C. FIG. 2 is a flowchart fordescribing photographing control, and FIGS. 3A to 3C are illustrativediagrams for describing the working in the first embodiment.

The control section 25 of the image pickup display apparatus 20 controlsthe image pickup control section 12 of the image pickup unit 10 throughthe communication sections 21 and 11. The image pickup control section12 is controlled by the control section 25 to drive the image pickupdevice of the image pickup unit 10. The picked-up image obtained byimage pickup performed by the image pickup device is imported into theimage pickup display apparatus 20 through the communication sections 11and 21. The control section 25 of the image pickup display apparatus 20performs predetermined signal processing on the imported picked-upimage, and thereafter sends the image subjected to the predeterminedsignal processing to the display section 23 to cause the display sectionto display a through image.

The user can perform photographing operation while referring to thethrough image displayed on the display section 23. The control section25 causes the display section 23 to perform operation display forphotographing operation and the like. In this case, the control section25 causes the display section to perform operation display according tothe state determination result by the state determination section 26.The state determination section 26 determines at least one of the statesof the image pickup display apparatus 20 and the image pickup unit 10,to output a state determination result to the control section 25.

For example, the state determination section 26 determines whether ornot the image pickup display apparatus 20 and the image pickup unit 10are separated from each other in step S1 in FIG. 2. When the imagepickup unit 10 is attached to the image pickup display apparatus 20, thecontrol section 25 selects a display form for an operation pattern 1 andperforms operation control according to the operation pattern 1, basedon the determination result by the state determination section 26 (stepS3).

FIG. 3A shows a display in the operation pattern 1. FIG. 3A shows astate where the display section 23 is provided on the substantiallyentire area of an operation surface of a case 20 a of the image pickupdisplay apparatus 20. On a display screen 23 a of the display section23, an image display area 31 for displaying a picked-up image 32 isprovided. In addition, on the display screen 23 a, a play button 35 a, amode button 35 b, and a photographing button 35 c, which are operationicons, are displayed.

When the user touches on the play button 35 a, the mode button 35 b, orthe photographing button 35 c, the control section 25 causes the imagepickup display apparatus 20 to shift to a play mode, a mode settingmode, or a photographing mode.

When the image pickup unit 10 is attached to the image pickup displayapparatus 20, it is expected that the user usually holds the case 20 awith both hands, and as shown in the example in FIG. 3A, the userperforms touch operation with both fingers (the thumbs, for example),i.e., both of a finger 41L of the left hand and a finger 41R of theright hand. Since the operation using the fingers of the both hands ispossible, the operation icons are arranged in arranging areas at up,down, right and left portions on the display screen 23 a, as shown inFIG. 3A.

The state determination section 26 determines whether or not theoperation surface of the image pickup display apparatus (device) 20faces upward and fixed in an up/down direction. For example, when theimage pickup display apparatus 20 is placed substantially horizontallyon an object according to the gravitational force, determination is madeon whether the image pickup display apparatus 20 is placed such that theoperation surface (touch surface) faces in the direction opposite to thedirection of the gravitational force. In the case where the image pickupdisplay apparatus 20 is not fixed, for example, in the case where theuser holds the image pickup display apparatus 20 with the left hand andholds the image pickup unit 10 with the right hand, the processingprocedure proceeds from the step S2 to step S4, and the control section25 performs display and control based on an operation pattern 2. Notethat the expression “fixed” is used in the above description. However,it may be determined that the image pickup display apparatus is in thefixed state, if the screen faces in a constant direction and is notlikely to vibrate in the direction. That is, the image pickup displayapparatus is not necessarily fixed in fact. In addition, theabove-described state can be called also as “on-the-desk placement”, buta board on which the image pickup display apparatus is placed is notlimited to a desk. The same operation can be performed also on theuser's lap, and depending on a situation, the case where the user abutsthe device on the wall and uses the device while sliding the device onthe wall is also included in the above-described state. Even when theuser uses the device in the above-described way, the wall functions as asupport, which allows the user to use the device easily in some cases.The above-described way of using can be described as “on-the-surfaceshift arrangement”, or “arranging on another object”. That is, the statedetermination section 26 determines whether or not the image pickupdisplay apparatus 20 is fixed to some extent on a surface of an objectby the gravitational force, a pressing force of some sort, or the like,and the touch operation or the like on the operation surface ispossible.

FIG. 3B shows a display in the operation pattern 2. A photographingbutton 36 a, a magnify button 36 b, and a reduce button 36 c, which areoperation icons, are displayed in an area other than the image displayarea 31 on the display screen 23 a.

When the user touches on the photographing button 36 a, the magnifybutton 36 b, or the reduce button 36 c, the control section 25 causesthe image pickup display apparatus 20 to shift to a photographing mode,a magnify mode, or a reduce mode.

In the case of this display form, it is expected that the user usuallyholds the case 20 a with one hand and holds the image pickup unit 10with the other hand, and performs the touch operation only with thefinger 41L of the left hand, for example, as shown in the example inFIG. 3B. Since the operation is performed using only the finger of theone hand, the operation icons are intensively arranged in the arrangingarea on the lower side on the display screen 23 a, as shown in FIG. 3B.

It is now supposed that the image pickup display apparatus 20 is placedon a table in the state where the image pickup unit 10 is detached. Inthis case, the processing procedure proceeds from the step S2 to stepS5. The control section 25 of the image pickup display apparatus 20performs display and control based on an operation pattern 3.

FIG. 3C shows a display in the operation pattern 3. A zoom button 37 a,an exposure button 37 b, and a focus button 37 c, which are operationicons, are displayed in an area other than the image display area 31 onthe display screen 23 a. In addition, also on the image display area 31,a photographing button 37 d and a play button 37 e are displayed.

When the user touches on the zoom button 37 a, the exposure button 37 b,the focus button 37 c, the photographing button 37 d, or the play button37 e, the control section 25 causes the image pickup display apparatus20 to shift to a zoom mode, an exposure mode, a focus mode, aphotographing mode, or a play mode.

In the case of this display form, it is expected that the user usuallyholds the image pickup unit 10 with one hand and places the image pickupdisplay apparatus 20 on a table, and the user performs the touchoperation with one hand, for example, with the fingers 42L, 43L or thelike of the left hand, as shown in the example in the FIG. 3C. Since theoperation using all the fingers of the one hand is possible, as shown inFIG. 3C, the operation icons are arranged in the arranging areas of theup, down, right and left portions on the display screen 23 a, and thenumber of the operation icons are larger than that in FIG. 3B.

In addition, since the slide operation with the fingers is relativelyeasy in this case, a slide bar 44 is also displayed on the displayscreen 23 a. The user performs the slide operation in the up/downdirection shown with the arrow 45, thereby capable of setting variouskinds of values.

When the control section 25 detects an operation for the displayaccording to each of the operation patterns 1 to 3, the control section25 performs photographing control according to the operation (step S7).

Note that the user interfaces shown in FIGS. 3A to 3C are examples, andvarious user interfaces can be set according to the states of the imagepickup display apparatus 20 and the image pickup unit 10.

In addition, the flowchart shown in the above-described FIG. 2 shows theexample in which the display form for accepting the user operation ischanged based on whether or not the image pickup display apparatus isseparated from the image pickup unit and placed on the table. However,the user interface may be configured to be changed depending on therespective states of the image pickup display apparatus and the imagepickup unit.

For example, FIG. 4 is a chart showing an example of a correspondencerelationship between the user interface and the respective states of theimage pickup display apparatus and the image pickup unit.

The example shown in FIG. 4 indicates the number of hands and fingerswhich can be used for operation, depending on whether the image pickupdisplay apparatus such as a smartphone is in the placed state or in theheld state (hand-held state) and whether the image pickup unit such as alens-style camera is in the stationary state or in the held state(hand-held state). The user interface may be appropriately set dependingon the number of available hands and fingers.

The example in FIG. 4 indicates that, in the case where the image pickupunit 10 is in the hand-held state, when the image pickup displayapparatus 20 is in the hand-held state, the image pickup displayapparatus 20 is required to be operated only with the thumb, on theother hand, when the image pickup display apparatus 20 is placed on thetable, the image pickup display apparatus 20 can be operated with fivefingers of one hand. In addition, FIG. 4 indicates that, in the casewhere the image pickup unit 10 is in the stationary state, when theimage pickup display apparatus 20 is in the hand-held state, the imagepickup display apparatus 20 is required to be operated with the thumb ofone hand and the five fingers of the other hand, on the other hand, whenthe image pickup display apparatus 20 is placed on the table, the imagepickup display apparatus 20 can be operated with the ten fingers of theboth hands.

Thus, in the present embodiment, a different user interface can beprovided depending on the state of at least one of the image pickupdisplay apparatus 20 and the image pickup unit 10, and the user canperform operation suitable for each of the states.

For example, when the image pickup unit 10 and the image pickup displayapparatus 20 are used in a state separated from each other, the imagepickup unit and the image pickup display apparatus can be used in amanner different from that in the case where the image pickup unit andthe image pickup display apparatus are unified, and there are situationswhere effective photographing is possible by using the separated imagepickup unit 10. For example, when an image of a dish is photographed, itis sometimes effective to photograph the dish not only from the frontside but also from the opposite side toward the front side. Theseparated image pickup unit 10 can be used for photographing in such asituation. In this case, it is expected that the user holds the imagepickup unit 10 such as a lens-style camera with one hand, and operates atablet PC or the like as the image pickup display apparatus 20 with theother hand. However, if the user holds the image pickup displayapparatus 20, the user has to hold the image pickup display apparatusand perform operation simultaneously with one hand, which results inunstable operation. Therefore, in such a case, it is more advantageousto use the image pickup display apparatus 20 in the state placed on atable or the like. In such a case, the present embodiment can provide auser interface which enables operation suitable for the placed state.This makes it relatively easy to operate the operation section 24 of theimage pickup display apparatus 20, which allows the user to concentrateon other photographing operations such as setting of a composition. As aresult, the user can perform stable photographing operation.

Second Embodiment

FIG. 5 is a block diagram showing the second embodiment of the presentinvention, and shows an example in which a lens-style camera and asmartphone are employed as the image pickup unit and the image pickupdisplay apparatus, respectively. FIG. 6 is an illustrative diagramshowing an appearance of the apparatus shown in FIG. 5. Note that atablet PC or a mobile phone may be employed instead of the smartphone.

In FIG. 5, a smartphone 61 corresponding to the image pickup displayapparatus 20 is provided with a camera communication section 62, alens-style camera 51 corresponding to the image pickup unit 10 in FIG. 1is provided with a communication section 55, and the smartphone 61 andthe lens-style camera 51 are configured to be able to communicate witheach other through respective communication sections 62 and 55.

As shown in FIG. 6, a case 61 a of the smartphone 61 is configured suchthat an attaching instrument 71 is detachably attached thereto, and theattaching instrument 71 is provided with an attaching portion 72 forattaching the lens-style camera 51. The lens-style camera 51 includes ona base end side thereof an attaching portion 51 b which is configured tobe attached to the attaching portion 72 of the attaching instrument 71by fitting, screwing, or the like.

The lens-style camera 51 is provided with an image pickup section 52having an optical system 52 a. The image pickup section 52 is providedwith an image pickup device, not shown, configured by a CCD, a CMOSsensor, or the like, and an object image is guided onto an image pickupsurface of the image pickup device by the optical system 52 a. Theoptical system 52 a includes, in a lens barrel 51 a, a focus lens whichis movable for setting a focus (focusing) state by focusing, and a zoomlens whose power is varied in the focus state. In addition, the opticalsystem 52 a includes a mechanical portion, not shown, which drives theselenses and a diaphragm. A control section 53 is configured to controlthe mechanical portion of the optical system 52 a to drive the focuslens, zoom lens and diaphragm.

The lens-style camera 51 is provided with an operation section 54 suchas an operation ring 54 a for performing setting operation of parametersfor photographing by a user, for example, parameters related tofocusing, zooming and diaphragm. The control section 53 detects the useroperation performed on the operation section 54, to perform controlbased on a result of the detection. The control section 53 is configuredby a CPU and the like, and controls the respective sections of thelens-style camera 51 based on the user operation performed on theoperation section 54 and a signal from the smartphone 61 to be describedlater.

A photographing control section 53 a generates a focus signal, a zoomsignal, and a diaphragm control signal, to drive and control thefocusing, zooming and diaphragm of the optical system 52 a. In addition,the photographing control section 53 a supplies a driving signal to theimage pickup device to control the image pickup of the object.

An image processing section 53 b receives a picked-up image from theimage pickup device, and performs predetermined image signal processing,for example, color adjustment processing, matrix conversion processing,noise removal processing, and other various kinds of signal processingon the picked-up image, and thereafter sends the image subjected to theimage signal processing to a recording section 56 and enables therecording section to record the image. As the recording section 56, anIC memory can be employed, for example. In addition, the imageprocessing section 53 b is configured to be able to transfer thepicked-up image to the smartphone 61 through the communication section55.

The communication section 55 can communicate with the cameracommunication section 62 provided in the smartphone 61 through apredetermined transmission channel. As the transmission channel, variouswired and wireless transmission channels, for example, a USB (UniversalSerial Bus) cable, a transmission channel for wireless LAN such as Wifi(Wireless Fidelity) can be employed. When communication is establishedbetween the lens-style camera 51 and the smartphone 61, the controlsection 53 is able to control the photographing in accordance with thecontrol section 65 of the smartphone 61 and transfer the picked-up imageand information related to the lens to the smartphone 61.

In the present embodiment, the lens-style camera 51 is provided with aposture determination section 57. The posture determination section 57is configured by a three-axis acceleration sensor, a gyro sensor, or thelike, and sends information such as the posture, rotation angle, etc.,of the lens-style camera 51 to the control section 53. Note that theposture determination section 57 may use a geomagnetic sensor togetherwith the above-described sensor. The control section 53 is configured tobe able to transmit the information (posture determination result) fromthe posture determination section 57 to the smartphone 61 through thecommunication section 55.

The smartphone 61 includes a built-in camera section 63. The built-incamera section 63 includes an image pickup device, not shown, which isconfigured by a CCD, a CMOS sensor, or the like, and configured suchthat the optical image of the object entered through a lens 63 a (seeFIG. 6) is guided to an image pickup surface of the image pickup device.

The control section 65 of the smartphone 61 is configured by a processorsuch as a CPU, for example, and controls the respective sections of thesmartphone 61. The control section 65 controls the built-in camerasection 63 and outputs a signal for driving the image pickup section 52of the lens-style camera 51. The control section 65 receives a picked-upimage from the built-in camera section 63, and also receives a picked-upimage from the lens-style camera 51. The control section 65 performspredetermined signal processing, for example, color adjustmentprocessing, matrix conversion processing, noise removal processing, andother various kinds of signal processing on the picked-up image from thebuilt-in camera section 63 and the picked-up image received from thelens-style camera 51.

The smartphone 61 is also provided with an operation section 66. Theoperation section 66 is configured by various kinds of operationsections such as switches, keys, software keyboard, or the like, whichare not shown, provided to the smartphone 61, and configured to generatean operation signal based on a user operation, and output the generatedoperation signal to the control section 65. The control section 65controls the respective sections based on the operation signal. Thecontrol section 65 is capable of performing processing related torecording and playing of the picked-up image. For example, the controlsection 65 is capable of performing compression processing on thepicked-up image subjected to the signal processing, transmitting thecompressed image to a recording section 64, and causing the recordingsection to record the image. As the recording section 64, various kindsof recording media such as an IC memory can be employed, and therecording section 64 is capable of recording image information, voiceinformation, and the like in the recording medium.

A display control section 67 performs various kinds of processingrelated to display. The display control section 67 receives thepicked-up image subjected to the signal processing from the controlsection 65, and is capable of sending the received image to a displaysection 68. The display section 68 has a display screen such as an LCD,and displays the image received from the display control section 67.Furthermore, the display control section 67 is capable of causing thedisplay section 68 to display various menus on the display screen. Thecontrol section 65 is capable of reading the picked-up image recorded inthe recording section 64 and performing expansion processing on the readpicked-up image. The display control section 67 sends the picked-upimage subjected to the expansion processing to the display section 68,thereby capable of playing the recorded image.

Also a touch panel, not shown, is provided as the operation section 66on the display screen of the display section 68. The touch panel iscapable of generating an operation signal corresponding to a position onthe display screen indicated by the user with a finger. The operationsignal is supplied to the control section 65. Thus, when the usertouches on or slides the display screen, the control section 65 iscapable of detecting various operations, for example, the positiontouched by the user, operation for closing and spreading two fingers(pinch operation), the sliding operation, the position reached by thesliding operation, the sliding direction, a period during which the usertouches the display screen, etc., and capable of performing processingcorresponding to the user operation.

Note that the display section 68 is disposed so as to cover thesubstantially entire area of the operation surface of the smartphone 61,for example, and the user is capable of checking the picked-up imagedisplayed on the display screen of the display section 68 at the time ofphotographing with the lens-style camera 51, and performingphotographing operation while checking the picked-up image.

Furthermore, in the present embodiment, the smartphone 61 includes aposture determination section 69. The posture determination section 69is configured by a three-axis acceleration sensor, a gyro sensor, or thelike, and sends information on the posture of the smartphone 61 to thecontrol section 65. Note that, the posture determination section 69 mayuse a geomagnetic sensor together with the above-described sensor.

The control section 65 is configured to change the operation interfacebased on at least one of the information (posture determination result)from the posture determination section 69 and the information on theposture determination result of the lens-style camera 51 received fromthe lens-style camera 51 through the camera communication section 62.That is, in the present embodiment, the graphical user interfaceprovided by the display section 68 and the operation section 66configured by the touch panel is controlled so as to be changedaccording to at least one of the states of the lens-style camera 51 andthe smartphone 61, based on the determination result from the posturedetermination sections 57 and 69.

Next, description will be made on the working of the present embodimentthus configured, with reference to FIG. 7 to FIGS. 16A-16D. FIG. 7 is aflowchart for describing the camera control in the lens-style camera 51and the control of a smartphone 61. Note that, in FIG. 7, the arrowsconnecting the camera control flow of the lens-style camera 51 and thecontrol flow of the smartphone 61 indicate that communication isperformed by processing.

The control section 53 of the lens-style camera 51 determines whether ornot the power source is turned on in step S31. When the power source isturned on, the control section 53 determines whether or not thephotographing mode is specified (step S32). When the photographing modeis specified, the control section 53 controls the image pickup section52 to cause the image pickup section to pick up an image of an object.The picked-up image obtained by the image pickup section 52 is importedby the control section 53 and a through image is obtained (step S33). Inaddition, the control section 53 obtains the posture determinationresult by the posture determination section 57, for example, thedetermination result of vibration in step S34.

On the other hand, the control section 65 of the smartphone 61determines whether or not a camera cooperative mode is specified in stepS11. When the camera cooperative mode is specified, the control section65 communicates with the lens-style camera 51 and generates a requestfor receiving a through image from the lens-style camera 15 in step S12.

Upon receiving the request for the through image from the smartphone 61,the control section 53 of the lens-style camera 51 establishescommunication in step S35, and thereafter transmits the through image tothe smartphone 61 through the communication section 15 (step S36). Thecontrol section 65 of the smartphone 61 receives the through image fromthe lens-style camera 51, sends the received through image to thedisplay control section 27 to cause the through image to be displayed instep S14.

Next, the control section 65 determines at least one of the states ofthe smartphone 61 and the lens-style camera 51 in order to provide agraphical user interface optimal for the photographing operation. Forexample, the control section 65 causes the built-in camera section 63 tobe activated and performs vibration determination using the output fromthe posture determination section 69 in step S15. Next, in step S16, thecontrol section 65 detects whether or not an image of a lens barrelconstituting the lens-style camera 51 is included in the picked-up imagefrom the built-in camera section 63. When the lens barrel image isincluded, the control section 65 determines that the lens-style camera51 is in the attached state, and sets the operation pattern 1 (stepS17).

FIG. 8 is an illustrative diagram showing an example of the picked-upimage from the built-in camera section 63. FIG. 8 shows a state where apicked-up image 81 obtained by the built-in camera section 63 isdisplayed on the display screen 68 a of the display section 68. A lensbarrel image 82 is included in the end portion of the picked-up image81. An image of the lens barrel 51 a of the lens-style camera 51 issometimes picked up by the built-in camera section 63, depending on theattaching position of the lens-style camera 51, the position of the lens63 a, and the like. The control section 65 detects whether or not thebarrel image 82 is included in the picked-up image by image processing,and is capable of determining that the lens-style camera 51 is attachedto the smartphone 61 when the barrel image 82 is included in thepicked-up image.

Furthermore, when the barrel image is not included in the picked-upimage, the control section 65 cannot determine whether or not thelens-style camera 51 is in the attached state based only on thepicked-up image from the built-in camera section 63. Therefore, thecontrol section 65 performs communication for connection determinationwith respect to the lens-style camera 51 in step S18. Also thelens-style camera 51 performs communication for connection determinationin step S37.

FIG. 9 is a flowchart showing an example a specific working flow of theconnection determination communication performed by the lens-stylecamera 51 in the step S37 in FIG. 7. Furthermore, FIG. 10 is a flowchartshowing a flow of the connection determination performed by the controlsection 65 using information obtained by the connection determinationcommunication.

The control section 53 of the lens-style camera 51 determines whether ornot a request for connection determination communication is generated instep S51 in FIG. 9, and determines whether or not the generated requestis a request for shake pattern in step S52. When determining that therequest for shake pattern is generated, the control section 53 transmitsinformation on the shake pattern which is information on vibration ofthe lens-style camera 51 based on the output from the posturedetermination section 57 (step S53).

On the other hand, the control section 65 of the smartphone 61determines a presence or absence of the camera communication between thesmartphone 61 and the lens-style camera 51 in step S61 in FIG. 10, toreceive information on the shake pattern from the lens-style camera 51.The control section 65 determines similarity between the received shakepattern and the vibration determination result (shake pattern) of thesmartphone 61 obtained in the step S15, and when determining that theshake patterns are similar to each other, the control section 65 shiftsthe processing to step S64 and when determining that there is nosimilarity, the control section 65 shifts the processing to step S63.

FIG. 11A and FIG. 11B are illustrative diagrams for describing thesimilarity determination of the shake patterns. FIG. 11A shows the shakepattern of the lens-style camera 51, and FIG. 11B shows the shakepattern of the smartphone 61. The control section 65 calculates changesin values of the respective shake patterns for each of the timings T1,T2, . . . in FIG. 11A and FIG. 11B, for example. The control section 65determines whether or not the changes are similar to each other. Whenthe changes in the values of the both shake patterns are similar to eachother, the control section 65 determines that the lens-style camera 51is connected to the case 61 a of the smartphone 61 (step S64). Inaddition, when the changes in the values of the both shake patterns arenot similar to each other, the control section 65 shifts the processingto the step S63. In the step S63, the control section 65 requeststransmission of image characteristics of the picked-up image obtained bythe lens-style camera 51.

The control section 53 of the lens-style camera 51 determines whether ornot the request for the image characteristics has been generated in stepS54 in FIG. 9. When the request for the image characteristics has beengenerated, the control section 53 transmits information on the imagecharacteristics to the smartphone 61 in step S55.

When receiving the information on the image characteristics from thelens-style camera 51, the control section 65 determines similaritybetween the received information on the image characteristics andinformation on image characteristics acquired from the picked-up imageobtained with the built-in camera section 63. When the information onthese image characteristics are similar to each other, the controlsection 65 determines that the field of view of the lens-style camera 51is similar to that of the built-in camera section 63, and the lens-stylecamera 51 is connected to the case 61 a of the smartphone 61 (step S64).When determining that there is no similarity in the shake patterns andin the image characteristics, the control section 65 determines that thelens-style camera 51 is not connected to the case 61 a of the smartphone61.

When determining the connection in the step S19 in FIG. 7, the controlsection 65 shifts the processing to the step S17 to set the operationpattern 1, and when determining that there is no connection, the controlsection 65 shifts the processing to the step S20 to perform display forseparation state. Note that the control section 53 of the lens-stylecamera 51 determines in step S38 whether or not the lens-style camera 51vibrates based on the output from the posture determination section 57,that is, determines whether vibration of the lens-style camera 51 isgenerated due to the lens-style camera 51 being held with the hand, orno vibration is generated due to the lens-style camera 51 being placedon a table or the like. When determining that the vibration of thelens-style camera is not generated, the control section 53 sets astationary mode in step S39, and transmits the determination resultregarding whether or not the lens-style camera is in the stationarystate to the smartphone 61.

In the operation pattern 1 in the step S17 in FIG. 7, the display andthe operation control shown in FIG. 3A are performed, for example. Onthe other hand, in the display for separation state in the step S20, theuser interface same as that in the operation patterns 2 and 3 shown inFIGS. 3B and 3C are set, for example.

FIG. 12 is a flowchart showing the working flow of the display forseparation state in the step S20 in FIG. 7. The control section 65determines whether or not the operation surface faces in the upwardvertical direction, that is, whether the user holds the smartphone 61with the hand in step S71 in FIG. 12, and determines whether thevibration amount in the vertical direction with respect to the operationsurface is smaller than a predetermined threshold, that is, determineswhether or not the smartphone 61 is placed on a table or the like instep S72. When at least one of the determination results in the step S71and step S72 is “NO”, the control section 65 determines that thesmartphone 61 is held with the hand, to shift the processing to stepS73, and when both of the determination results in the step S71 and stepS72 are “YES”, the control section 65 determines that the smartphone 61is placed on a table or the like, to shift the processing to step S74.

In the step S73 and step S74, the control section 65 determines whetheror not the lens-style camera 51 is in the stationary state. Whendetermining that the smartphone 61 is in the hand-held state and thelens-style camera is also in the hand-held state, the control section 65shifts the processing to step S75. In this case, since both hands of theuser are occupied for holding the smartphone and the lens-style camera,the control section 65 sets the operation pattern 2 shown in FIG. 3B,for example (step S75).

When determining that the smartphone 61 in the hand-held state and thelens-style camera is in the stationary state, the control section 65shifts the processing to step S76. In this case, since only one hand ofthe user is occupied for holding the smartphone, operation of thesmartphone 61 using all the fingers of the other hand which does nothold the smartphone 61 is possible, and the control section 65 sets theuser interface suitable for the operation.

In addition, when determining that the smartphone 61 is in the placedstate and the lens-style camera is in the hand-held state, the controlsection 65 shifts the processing to step S77. In this case, since onehand of the user is occupied for holding the lens-style camera, thecontrol section 65 sets the operation pattern 3 shown in FIG. 3C, forexample (step S77).

Furthermore, when determining that the smartphone 61 is in the placedstate and also the lens-style camera is in the stationary state, thecontrol section 65 shifts the processing to step S78. In this case,since both hands of the user are not occupied for holding the smartphoneand the lens-style camera, operation of the smartphone 61 using all thefingers of both hands is possible, and the control section 65 sets theuser interface suitable for the operation.

In the step S21 in FIG. 7, the control section 65 determines whether ornot an instruction for photographing has been given by the user. Whenthe photographing operation is performed, the control section 65requests the lens-style camera 51 to photograph the image displayed asthe through image (step S22). The control section 53 of the lens-stylecamera 51 determines whether or not communication for instructing thephotographing is performed in step S40, performs photographing inaccordance with the instruction from the control section 65 of thesmartphone 61, and transfers the picked-up image to the smartphone 61(step S41). The control section 65 performs signal processing on thepicked-up image from the lens-style camera 51, and thereafter transmitsthe signal-processed image to the recording section 64, to cause therecording section 64 to record the image (step S23).

The control section 53 of the lens-style camera 51 also determineswhether or not the instruction for photographing has been given byoperating the operation section 54 of the lens-style camera 51 (stepS42). When the instruction for photographing is given by the operationsection 54, the control section 53 performs photographing in step S43,and then causes the recording section 56, for example, to record thepicked-up image (step S44).

Next, as an example of use in the present embodiment, description willbe made on an example in which a dish is photographed with reference toFIG. 13A, FIG. 13B, and FIG. 14 to FIGS. 16A-16D. FIGS. 13A and 13B areillustrative diagrams for describing a state of photographing. In FIGS.13A and 13B, a serving plate 92 is placed on a table 91, and a fish dish93 is on the serving plate 92. A user 94 holds the lens-style camera 51with a right hand 95R. The smartphone 61 is placed on the table 91, withthe display screen 68 a facing upward. In this case, the state of thesmartphone corresponds to the state expressed as what is called“on-the-surface arrangement” or “on-the-desk placement”. However, if thescreen faces in a constant direction and is not likely to vibrate in thedirection, the state may be determined as the above-described state, andthe smartphone is not necessarily fixed and the board on which thesmartphone is placed is not limited to the table shown in the drawing ora desk. The same operation can be performed also on the user's lap, anddepending on a situation, there is case where the user uses thesmartphone while abutting the smartphone on the wall. The case where theuser uses the smartphone while sliding the smartphone along the surfaceof the wall is also included in the above-described state. Even when theuser uses the smartphone in the above-described way, the surfacefunctions as a support, which allows the user to use the smartphoneeasily in some cases. That is, the above-described way of using can bedescribed as “on-the-surface shift arrangement”, or “arranging onanother object”. That is, the smartphone 61 is fixed to some extent on asurface of an object by the gravitational force, a pressing force ofsome sort, or the like, and the touch operation on the display screen 68a is possible.

The user 94 can check the through image displayed on the display screen68 a of the smartphone 61 while moving the right hand 95R which holdsthe lens-style camera 51. The user 94 tries to determine the compositionby appropriately moving the right hand 95R while checking the throughimage. The arrow in FIG. 13A indicates that the user shifts the righthand 95R to arrange the lens-style camera 51 at various positions.

As shown in FIG. 13A, when photographing the dish, if the smartphone 61having the display section 68 is placed on the table 91, the user 94 hasonly to look at one location where the display screen 68 a is positionedeven when the user aims at an object with the lens-style camera 51 fromvarious angles. If the user 94 decides the composition by referring tothe image on the display screen 68 a, the user 94 then has only toperform camera operation by performing touch operation on the displayscreen 68 a with a left hand 95L, as shown in FIG. 13B. In this case,since the smartphone 61 is placed on the table 91, the user 94 does nothave to support the smartphone 61, which enables the user to easilysupport the lens-style camera 51.

That is, the examples shown in FIGS. 13A and 13B can provide the userinterface in the above-described operation pattern 3. In this case, thecontrol section 65 of the smartphone 61 performs control in theoperation pattern 3 in accordance with the working flow shown in FIG.14, for example FIGS. 15A to 15C are illustrative diagrams fordescribing an example of the display in this case.

In step S71 in FIG. 14, the control section 65 determines whether or notan input mode has already been selected. Examples shown in FIG. 14 andFIGS. 15A to 15C show two input modes including a default mode and akeyboard mode in which software keyboard is displayed are possible, anddetermination is made on whether or not these input modes are specifiedin the step S71.

When the input mode is not selected, the control section 65 displays, instep S72, a mode switch (SW) for selecting the input mode. FIGS. 15A and15B show such a state, and on the display screen 68 a provided on theoperation surface side of the case 61 a of the smartphone 61, a defaultbutton 105 and a keyboard button 106 are displayed as the mode switch(SW) for the user to specify either the default mode or the keyboardmode. Note that, as shown in FIGS. 15A and 15B, a picked-up image 101received from the lens-style camera 51 is displayed on the displayscreen 68 a. In addition, also a photographing button 102 and a playbutton 103 are displayed on the display screen 68 a.

When the user touches on the photographing button 102 or the play button103 with a finger 107, the control section 65 instructs the lens-stylecamera 51 to perform photographing or playing. In addition, when theuser touches on the default button 105 or the keyboard button 106 withthe finger 107, the control section 65 sets the interface to the defaultmode or the keyboard mode. FIGS. 15A and 15B show the state where theuser touches the buttons 105 and the state where the user touches thebutton 106, respectively, by displaying the frames of the respectivebuttons with thick frames.

When the user selects the default button 105 with the finger 107, thecontrol section 65 shifts the processing from the step S73 to the stepS74, to set the default mode. In this case, the display and operationcontrol shown in FIG. 3C are performed, for example. In addition, whenthe user selects the keyboard button 106 with the finger 107, thecontrol section 65 shifts the processing from the step S73 to step S75,to set the keyboard mode. In this case, the display and operationcontrol shown in FIG. 15C are performed, for example.

The user interface shown in FIG. 15C allows the user to control thecamera operation with keyboard input, and in addition to the picked-upimage 111 received from the lens-style camera 51, a key display 112 forkeyboard input is shown on the display screen 68 a. The key display 112is configured to allow a plurality of kinds of characters to bespecified with one key and enable a character string to be specified bythe number of touching a key or what is called a flicker input in whichthe user slides the finger after touching the key.

The user can specify the control mode of the camera operation throughkeyboard input using the finger 107. The setting performed by the useris shown in a setting display section 114. The example shown in FIG. 15Cshows that the user inputs “rotation” with the finger 107. The userinputs “rotation”, to thereby cause the control section 65 to set arotation mode to be described later. Note that the smartphone 61includes a database, not shown, in which a plurality of characterstrings are described so as to correspond to one camera control mode,and the user can specify a desired control mode by inputting appropriatecharacter strings related to the desired control mode. Note that a backbutton 113 for undoing processing is also displayed on the displayscreen 68 a.

Such a keyboard input operation using a software keyboard is an inputmethod suitable for a tablet PC or a smartphone. In addition, the userhas only to perform operation on the smartphone 61 placed on the table91, which allows the control mode to be specified relatively easy. Inthe method of selecting a control mode from a menu display constitutedof a plurality of hierarchies to decide the control mode, if the mode tobe selected is set in a deep hierarchy, for example, the mode specifyingoperation by the user is relatively cumbersome in some cases. Even insuch a case, in the example shown in FIG. 15C, it is possible to easilyspecify the mode with keyboard input operation.

When the input mode is decided, the control section 65 shifts theprocessing to step S81 to detect presence or absence of buttonoperation. When the user operates various buttons other than the keydisplay 112 with the finger 107, the control section 65 shifts theprocessing to step S82 to perform control corresponding to the buttonoperation. For example, when the user touches the position of theexposure button 37 b in FIG. 3C, the control section 65 shifts the modeto an exposure setting mode.

On the other hand, when the user touches the key display 112, thecontrol section 65 shifts the processing from step S83 to step S84, todetermine the control mode according to the key input operation by theuser (step S84). As shown in FIG. 16A, it is now supposed that the userperforms an operation for touching the position of the setting displaysection 114 in the state where the characters “rotation” is displayed onthe setting display section 114. Then, the control section 65 determinesthat the key operation by the user is for instructing the rotation modein the step S4 to shift the mode to the rotation mode. Note that, whenthe instructed mode is not a rotation mode, the control section 65executes another mode specified in the step S89. For example, as theother mode, various modes such as zoom mode, exposure mode, focus mode,and art filter mode can be considered.

In the photographing method shown in FIGS. 13A and 13B, the user holdsthe lens-style camera 51 with one hand and photographs the object fromvarious angles. Therefore, the orientation of the image pickup device inthe lens-style camera 51 is not necessarily in the horizontal orvertical direction. Specifically, when the lens-style camera 51 isconfigured by a cylindrical-shaped case, the user sometimes cannotrecognize the orientation (rotation direction in the vertical plane) ofthe picked-up image unless the user sees the picked-up image on thedisplay screen 68 a. In this case, it is sometimes more preferable forthe user to change the orientation of the picked-up image by performingimage processing on the smartphone 61 side than to change theorientation of the lens-style camera 51 with the hand holding thelens-style camera 51. The rotation mode is for performing imageprocessing for changing the rotation direction of the picked-up image.

FIGS. 16A to 16D are illustration drawings for describing the rotationmode.

When the rotation mode is set according to the button operation or thekey input operation, the control section 65 shifts the processing fromstep S85 to step S86, to determine the initial position of thesmartphone 61. Next, the control section 65 determines the rotation ofthe smartphone 61 to calculate the rotation angle.

For example, in the state where the user touches the position of thesetting display section 114 with the finger 107 a as shown in FIG. 16A,the user holds the both side surfaces of the case 61 a of the smartphone61 between the finger 107 b and another finger not shown, as shown inFIG. 16B and rotates the smartphone within the surface (within thehorizontal surface) of the table 91 with the rear surface of the case 61a contacting the table 91. The initial position of the rotation isdetermined in the step S86, and the rotation amount is calculated instep S87. Note that the operation (rotation operation) method forspecifying the rotation amount by the user can be appropriately set. Forexample, the control section 65 may determine that the rotationoperation is finished when the user moves the finger 107 a away from theposition of the setting display section 114, and calculate the rotationangle at which the rotation operation is finished. Furthermore, thecontrol section 65 determines the rotation amount by the number ofrepeat of the rotation operation in one direction and in the directionopposite to the one direction by the user or a speed of the rotation.

FIG. 16C shows an example of screen display when the user performs therotation operation. In the example in FIG. 16C, the control section 65allows an image obtained by rotating a display area 115 of the picked-upimage 117 by the rotation amount corresponding to the rotation operationby the user to be displayed on the display screen 68 a. If the rotationoperation is finished in this state, the control section 65 generates apicked-up image 116 obtained by rotating the picked-up image 117 by therotation amount corresponding to the rotation operation by the user(step S88), to cause the generated picked-up image 116 to be displayedon the display screen 68 a.

When a predetermined period elapses from the state displaying the imageas shown in FIG. 16C, or a predetermined operation is performed, therotation mode is finished and the display returns to the one shown inFIG. 13A, for example. When the photographing button 102 is operated inthis state, the picked-up image 116 shown in FIG. 16D is recorded in therecording section 64.

The present embodiment is thus capable of detecting the states of thelens-style camera and the smartphone, and providing the user interfacesuitable for at least one of the states. Such a configuration improvesthe operability, and the user can photograph an object in a desiredcomposition with a simple operation. When the lens-style camera and thesmartphone are used in the state separated from each other, inparticular, the photographer does not have to move in accordance withthe position of the lens-style camera. The photographer can capture theobject in various compositions, and easily perform photographingoperation by placing the smartphone on the table, and the like. Thus, ifthere are two devices to be operated with two hands of a human being, itis considered that there are various cases such as a case where the twodevices are operated with both hands, respectively, a case where one ofthe devices is operated with both hands or with one hand, etc. However,there is not known a technique for controlling the devices by taking thestates of the devices in the above-described cases into fullconsideration. According to the present invention, it is possible tosurely achieve a simple operation using an optimal operation methodunder various situations. That is, according to the present invention,it is possible to detect the respective states such as holding state orplaced-on-the-desk state of the two cooperative devices which are usedby a user who operates the devices generally with two hands, by usingthe shaking determination function or the posture determination functionprovided to each of the devices, decide an operation optimal to thedevices depending on the respective usage states of the devices, controlthe display, determine the operation, and perform control of simple andconvenient cooperation of the devices. The present invention effectivelyuses the touch panel and display control of the devices.

Furthermore, in each of the embodiments of the present invention,description has been made by taking the digital camera as an example ofthe device for photographing. However, it is needless to say that thecamera may be a lens-style camera, a digital single-lens reflex camera,a compact digital camera, a camera for moving picture such as a videocamera and a movie camera, and also may be a camera built in a mobileinformation terminal (PDA: Personal Digital Assist) such as a mobilephone and a smartphone. Furthermore, the camera may be an industrial ormedical optical device such as an endoscope and a microscope, or amonitoring camera, a car-mounted camera, or a stationary camera, forexample, a camera attached to a television, a personal computer, or thelike. It is needless to say that the present invention can be applied toan observation apparatus and an inspection apparatus which observevarious objects from various angles. When inspection or observation isperformed, it is more convenient if a plurality of devices can bemechanically connected or separated, or a way of holding the devices canbe changed, depending on an object to be observed or inspected.Therefore, the present invention can be utilized in wide variety offields.

The present invention is not limited to each of the embodiments as itis, but can be embodied by modifying constituent elements in thepractical stage within a range without departing from the gist of theinvention. Furthermore, various inventions can be formed byappropriately combining the plurality of constituent elements disclosedin each of the embodiments. For example, among all of the constituentelements shown in the embodiments, some constituent elements may beremoved. Furthermore, the constituent elements from differentembodiments may be appropriately combined.

Note that, in the claims, specification, and working flows in thedrawings, even if description was made by using the expressions “first”,“next” and the like, for convenience sake, it does not mean thatprocessing steps have to be executed in this order. In addition,needless to say, each of the steps constituting the working flow may beappropriately omitted regarding the part which does not affect theessential part of the invention.

In addition, among the techniques described above, many of the controlsand functions described mainly in the flowcharts can be set by aprogram, and the above-described controls and functions can be executedby reading and executing the program by a computer. The entirety or apart of such a program can be recorded or stored as a computer programproduct in a portable medium such as a flexible disk, CD-ROM and anonvolatile memory, or a storage medium such as a hard disk and avolatile memory, and can be distributed or provided at the time ofproduct shipment or through a portable medium or telecommunicationsline. A user can easily use the image pickup display apparatus anddisplay method according to the embodiments by downloading the programthrough a communication network to install the program into a computer,or by installing the program from a recording medium into a computer.

What is claimed is:
 1. An image pickup display apparatus comprising: acommunication section that is capable of receiving a picked-up imagefrom an image pickup unit; a display section that displays the picked-upimage received by the communication section; an operation section forcontrolling image pickup by the image pickup unit; a state determinationsection that determines a state of a case in which the communicationsection, the display section and the operation section are housed; and acontrol section that receives a result of determination performed by thestate determination section, to change a user interface provided by thedisplay section and the operation section based on whether or not thecase is arranged on another object.
 2. The image pickup displayapparatus according to claim 1, wherein the state determination sectiondetermines a state of the image pickup unit, and the control sectionreceives the result of determination performed by the statedetermination section, and changes the user interface provided by thedisplay section and the operation section based on whether or not thecase is arranged on the other object and whether or not the image pickupunit is arranged in a stationary state.
 3. The image pickup displayapparatus according to claim 1, wherein the state determination sectiondetermines whether or not the case and the image pickup unit areseparated from each other, and the control section changes the userinterface provided by the display section and the operation sectionbased whether or not the case is arranged on the other object andwhether or not the case and the image pickup unit are separated fromeach other.
 4. The image pickup display apparatus according to claim 1,wherein the state determination section determines that the case isarranged on the other object when an operation surface of the operationsection is fixed so as to face upward.
 5. The image pickup displayapparatus according to claim 2, wherein the state determination sectiondetermines that the case is arranged on the other object when anoperation surface of the operation section is fixed so as to faceupward.
 6. The image pickup display apparatus according to claim 3,wherein the state determination section determines that the case isarranged on the other object when an operation surface of the operationsection is fixed so as to face upward.
 7. The image pickup displayapparatus according to claim 4, wherein the state determination sectiondetermines that the case is arranged on the other object when vibrationof the operation surface of the operation section in a verticaldirection is smaller than a predetermined threshold.
 8. The image pickupdisplay apparatus according to claim 5, wherein the state determinationsection determines that the case is arranged on the other object whenvibration of the operation surface of the operation section in avertical direction is smaller than a predetermined threshold.
 9. Theimage pickup display apparatus according to claim 6, wherein the statedetermination section determines that the case is arranged on the otherobject when vibration of the operation surface of the operation sectionin a vertical direction is smaller than a predetermined threshold. 10.The image pickup display apparatus according to claim 1, wherein theoperation section is constituted of a touch panel disposed on thedisplay section, and the control section changes a number of operationicons to be displayed on the display section based on whether or not thecase is arranged on the other object.
 11. The image pickup displayapparatus according to claim 1, wherein the operation section isconstituted of a touch panel disposed on the display section, and thecontrol section changes an arranging area of operation icons to bedisplayed on the display section based on whether or not the case isplaced on the other object.
 12. The image pickup display apparatusaccording to claim 1, wherein the operation section is constituted of atouch panel disposed on the display section and a software keyboarddisplayed on the display section.
 13. The image pickup display apparatusaccording to claim 1, wherein the control section rotates the picked-upimage from the image pickup unit in accordance with rotation of thecase, when the case is arranged on the other object.
 14. An image pickupdisplay method comprising: receiving, by a communication section, apicked-up image from an image pickup unit; displaying, by a displaysection, the picked-up image received by the communication section;determining, by a state determination section, a state of a case inwhich an operation section for controlling image pickup by the imagepickup unit, the communication section, and the display section arehoused; and receiving a result of determination performed by the statedetermination section, to change a user interface provided by thedisplay section and the operation section, by a control section, basedon whether or not the case is arranged on another object.
 15. Anon-transitory computer readable recording medium which stores a programfor causing a computer to execute procedures of: receiving, by acommunication section, a picked-up image from an image pickup unit;displaying, by a display section, the picked-up image received by thecommunication section; determining a state of a case in which anoperation section for controlling image pickup by the image pickup unit,the communication section, and the display section are housed; andchanging a user interface provided by the display section and theoperation section based on whether or not the case is arranged onanother object.